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ABSTRACT: In recent years, technology has been playing a transformative role in the field of built environment, 
architecture, and construction education. It can be argued that the emergence of digital technologies has 
revolutionised the approach to teaching and learning in higher education in these fields. Digital technologies, 
such as Artificial intelligence (AI), additive manufacturing, robotics, 3D laser scanners, and Immersive Realities 
(IR), have played a crucial role in enhancing sustainability and efficiency in the industry. However, the 
opportunities provided by the use of these technologies (as a single tool or combined) in higher education and 
within the field of Architecture, Engineering, and Construction (AEC) are still relatively unexplored. To address 
this gap, this work presents a novel pedagogical framework aimed to enhance students’ literacy on emerging 
technologies, and increase their criticality, and understanding of professional practices along with the related 
ethical challenges. Furthermore, to assess its effectiveness regarding the integration of immersive VR technologies 
in the teaching practice, a learner-centred evaluation approach is proposed, based on the collection and 
correlation of both qualitative and quantitative data. Concerning the former, a dedicated questionnaire is 
developed to collect students’ subjective feedback. For the latter, a method for tracking their use of space in the 
virtual environment is discussed. Both the immersive pedagogical framework and evaluation approach presented 
in this work will be implemented in diverse architecture and civil engineering master classes in Australia and in 
Italy, and their comparative outcomes and validation will be the object of future joint contributions. 


KEYWORDS: Digital and Immersive Pedagogy, Digital Technology; Architectural Higher Education; Immersive 
Reality, Evaluation, Questionnaire, Spatial tracking. 


1. INTRODUCTION 


Over the past decades, there has been a rapid growth in urbanisation and development of urban areas across the 
world. This has resulted in a considerable increase in the demand for new buildings, structures and infrastructures, 
which, in turn, brings along a number of environmental, social and economic challenges. The nature of such 
challenges has become increasingly complex, so much so, that the traditional and conventional methods of 
construction cannot address them. Emerging technologies and digital tools have been increasingly applied in this 
field to address such challenges and achieve a more sustainable, safe, efficient and optimized practice (Alsafouri 
& Ayer, 2018; Alsafouri & Ayer, 2019; Ardito et al., 2019; Davila Delgado, Oyedele, Beach, et al., 2020; Davila 
Delgado, Oyedele, Demian, et al., 2020; Fazel & Izadi, 2018; Hajirasouli & Banihashemi, 2022; Hajirasouli et al., 
2022; Hamzeh et al., 2019; Mandolla et al., 2019; Moon et al., 2015; Nafors et al., 2020; Rohani et al., 2014; 
Valero et al., 2015). Among such technologies, Immersive Realities (IR) such as Virtual Reality (VR) and 
Augmented Reality (AR) have proven to be very advantageous in various areas of built environment. When 
considering architecture discipline and the required spatial qualities, capabilities and understanding required for 
it, VR seems to be a more appropriate tool to incorporate in its pedagogy, with multiple advantages (Getuli et al., 
2020; Hajirasouli & Banihashemi, 2022; Hajirasouli et al., 2022; Rahimian et al., 2019). Despite the emphasis that 
have been made by a number of studies and scholars regarding the development of digitally enhanced and 
technology-integrated teaching methods (Aydin & Aktaş, 2020; Bashabsheh et al., 2019; Ceylan, 2021; Hajirasouli 
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& Banihashemi, 2022; Shirazi & Behzadan, 2015), a number of studies conducted by authors have identified that 
the current teaching and learning practices in the AEC higher education does not adequately embrace and 
incorporate such digitally enhanced methods and, therefore, are yet to respond to the industry’s demand in this 
area (Getuli et al., 2020; Hajirasouli & Banihashemi, 2022; Hajirasouli et al., 2022; Pour Rahimian et al., 2019). 
More importantly, when developing such pedagogies, how would this affect students' perception of their education 
and impact their teaching and learning experience. 


This study aims to provide an in-depth understanding of student's needs and requirements and, more importantly, 
their perception of the integration of new technologies in their courses and curriculum. For this purpose, in Section 
2, a theoretical framework was developed to establish a novel immersive pedagogy. Building on the constructivist 
assumption of active learning for a digitally enabled pedagogy, a problem-based learning process fostered by 
immersive technologies is conceptualized. Furthermore, to assess the framework's effectiveness, an original mixed 
method for the evaluation of VR-based teaching experiences for AEC students is discussed in Section 3. This 
comprises a dedicated questionnaire to be administered to the students after the immersive learning experiences to 
collect their subjective perceptions and feedback. Complementarily, the tracking of their virtual position and its 
restitution in the form of contextualized heatmaps is proposed to objectively evaluate their use of the virtual 
environment in relation to the learning objectives. In Section 4, the implementation plan of the proposed 
pedagogical framework and evaluation method is then presented with reference to the architecture and civil 
engineering master classes which will be involved both in Australia (Western Sydney University) and Italy 
(University of Florence). Eventually, the discussion of the limitations and outlook of this study is provided in 
Section 5. 


2. PEDAGOGICAL FRAMEWORK 


To develop the proposed theoretical model a critical literature review was conducted, including the relevant works 
related to previously developed models in this area. From their analysis, constructivism philosophy emerges as the 
most appropriate teaching philosophy to be adapted to this framework, due to the constructive nature of immersive 
technologies and their implementation in teaching and learning activities. In fact, constructivism philosophy is 
correlated with creating and constructing new knowledge based on the learner’s already existing knowledge, 
therefore implying an active and continuous participation in the process of learning (Behzadan et al., 2015; 
Behzadan et al., 2011; Biggs & Tang, 2007; Bruning et al., 1999; Lord, 1999; Luo & Mojica Cabico, 2018; Tynjala, 
1999; Von Glasersfeld, 1995). Hence, the constructivist approach was used as the principal philosophy for the 
developed model. 


Choosing the right approach for implementing this model was the next step of this work. It is suggested that the 
concept of digital pedagogy does not only reflect upon using digital tools and technologies, rather, it is also about 
cautiously considering their effects and implications from a critical pedagogical point of view. Therefore, the 
decision about their integration within the teaching approach or not, depends on the desired learning outcome of a 
course or subject (Anderson, 2020; Barber et al., 2015; Croxall, 2013; James & Pollard, 2011). Problem-based 
learning approach was also used in this model as an integral part of the application and integration of immersive 
technologies. The selection of this approach was also due to its suitability for complex real-world situations where 
there is no right or wrong answer to the problem (Barber et al., 2015; Savin-Baden, 2007; Word, 2003), which is 
the main focus of AEC discipline and industry. This approach helps students to work collaboratively in groups, to 
identify the problems and gaps, and to develop solutions and knowledge, through self-directed processes (Barber 
et al., 2015; Savin-Baden, 2007; Word, 2003). 


Eventually, immersive learning was also used as the last stage of this model. Using this method, the creation and 
construction of knowledge occurs through virtual immersion into a context, dialogue and/or situation. Immersion 
occurs in two different ways: immersion through narrative (cognitive aspects), and immersion through 
technological devices (technical aspects). This study focuses on immersion through technical devices, hence 
requires various tools and technologies, such as AR, VR, and Virtual Learning Environment (VLE). The choice 
of tools and technologies in this model depends upon the level of immersion required for a learning objective. VR, 
which is the subject of this study, is mainly being used when a fully immersive experience and a sense of presence 
in the virtual environment is required for the learning process. 


3. IMMERSIVE VR LEARNING EXPERIENCE EVALUATION 


The proposed pedagogical approach theorises the beneficial impact that the adoption of immersive visualization 
technologies can provide in AEC master classes. To support this claim and assess its effectiveness in upcoming 
case study implementations (see Section 4), an original method for the evaluation of immersive VR learning 
experiences is developed, considering both qualitative subjective data and quantitative objective observations. As 
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shown in Figure 2, a questionnaire to be administered to the student after the immersive experience and divided 
into five inquiry areas is provided. Besides, the student’s use and understanding of the virtual environment in 
relation to the learning objectives is evaluated with the acquisition of their spatial track during the VR experience 
and its sequential visualization of a BIM environment in the form of a heatmap. 


Constructivism 


Learning 


Digital Learning 


Making a decision not to 
use digital technologies 


Immersive Learning 


Immersion using 
narrative 


Virtual Learning 
Environment 


Augmented Reality 


Fig. 1: Pedagogical framework designed for BIM-enabled VR-based technology application into architectural 
design studio. 
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Fig. 2: Immersive VR learning experience evaluation approach — Data schema. 


268 


3.1 Post-experience evaluation questionnaire 


To collect students’ subjective feedback, a dedicated questionnaire has been developed to be administered directly 
after the immersive learning experience. First, data necessary to classify, analyse and classify the questionnaire 
responses based on students’ demographics are collected. Then the experience is evaluated against the student’s 
perceptions, concerning: the engagement with the virtual environment (immersivity); the reproduction of the virtual 
contents (contents effectiveness, e.g., graphical realism compared to the objective of the session); the gained self- 
efficacy and competency (learning experience effectiveness); the opinion on suggested additional features that 
could be implemented in the experience and that were currently missing (future developments). For the cited 
criteria the responses are collected according to a scoring system ranging from one to five, where five correspond 
to the maximum agreement with the statement or satisfaction with the experience. Furthermore, an open textual 
field (suggestions) is provided to enable the collection of personal remarks, impressions, and ideas, involving and 
empowering the student in the improvement and evolution of a learning approach based on ever-changing 
technologies and that will need to keep up to learners’ expectations and needs to be future proof. In Table 1 the 
data type and the description and purpose of data collection are reported along with explanatory examples. 


Table 1: Questionnaire data type description and prompt example 


Description Example 


Data type 


Questionnaire ID: (Text) 


Data necessary to analyse and classify the responses collected through the Immersive experience: (Text) 


ID & demographics 


questionnaires with reference to the occurred immersive learning 
experiences, and to cluster their results also with reference to the 


students’ demographics (anonymized). [Various] 


Date: (yyyy/mm/dd) 

Age: (Number) 

Gender: (Multiple options) 
Course: (Text) 


Questions related to the student’s ability to get immersed in the virtual 


experience. They are useful for assessing how engaged the students were 


How much did you like the 


Immersivity in the virtual world and to evaluate their satisfaction in terms of both ease experience? 
of use and comfort (also related to the possible onset of symptoms of [min 1; max 5] 
"motion sickness"). [Single rating — Likert scale] 
T a ; . a How efficiently and clearly does the 
Questions regarding the virtual representation of the learning contents, ; 
Contents ; p . A i content of VR help you to perceive 
i aimed at evaluating the effectiveness of the experience against the : k 
effectiveness the discussed subject better? 


objectives of the session. [Single rating — Likert scale] 


[min 1; max 5] 


Learning experience 


Questions pertaining to the overall effectiveness of the experience, aimed 


at investigating the actual usefulness of the immersive VR learning 


Do you think this experience is 


useful in understanding the qualities 


effectiveness session compared to traditional methods, especially concerning the of the designed spaces? 
learning objectives. [Single rating — Likert scale] [min 1; max 5] 
Questions concerning the introduction of new features (e.g., content Would you like to be able to grasp 
Future animations, audio and visual effects, etc.) or virtual content aimed at and interact with objects in the VR 
developments enhancing the immersive VR learning experience through the inclusion of | environment? 
greater realism and/or interactivity. [Single rating — Likert scale] [min 1; max 5] 
; Open questions to collect personal impressions and ideas from the : 
Suggestions Suggestions 


students. [Text box] 


3.2 Student’s spatial tracking and use of space visualization 


In the previous paragraph, it has been discussed how the students are actively involved in providing data for a 
qualitative evaluation. Here, the second, complimentary, data acquisition method is presented which is based on 
the objective observation of the virtual positions covered by the students throughout the experience. In turn, this 
involves capturing the position of the student in the VR environment during the entire course of the simulation in 
order to analyse their actual use of the tridimensional space. For this purpose, the student's virtual position shall 
be recorded with a data acquisition rate of at least 1 Hz. The 3D point sequence resulting from this process shall 
be transferred in a BIM environment and converted for the generation of heatmap visualization, within which the 
position of the student is represented, weighted by the time spent, with a colour gradient. In this way, the relevance 
of different areas of the experience can be evaluated based on the time spent in certain virtual locations by the 
students. As with the questionnaire’s development, the student spatial tracking and visualization procedure to be 
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performed during the learning experience is explained with reference to the collected data type in Table 2. 


Table 2: Student’s spatial tracking and use of space visualization characteristics 


Data type Description Example 


ID & demographics See Table 1. See Table 1. 


The duration of the experience from start to finish, excluding a possible 


: ? tutorial or time required to the student to get used with the immersive VR 
Learning experience 


F: system and controllers. This is necessary to allow for the later heatmap Duration: (number) sec 
uration 
visualization of the student use of the virtual space weighted on the 
overall elapsed time. [time in seconds] 
The student position in the virtual environment is collected as 3D point . f 
J 4 i i . Colour gradient representation of 
. with 1 Hz frequency. The corresponding spatial track is then graphically 
Student spatio- 5 i $ the student’s followed path 
represented against a model of the experienced environment (e.g., BIM : f 
temporal track (weighted by time) 


model) both as a 3D path (polyline) and with an heatmap representation, : 
: . ‘ : i i i [min. green, max. purple] 
with the colour gradient weighted on the time spent in a certain location. 


4. FRAMEWORK IMPLEMENTATION PLAN 


The theoretical approach of this paper is built upon previous research projects, undertaken by authors in Australia 
and Italy, to create an innovative approach and prototype protocol for the design, delivery and evaluation of a 
number of subjects for AEC students in higher education, based on an interactive and immersive learner-centred 
approach. The outcome of this work will be implemented in a number of course subjects at the University of 
Florence, Italy and Western Sydney University, Australia. The nominated subjects for the implementation of this 
model are reported in Table 3. 


Table 3: Immersive VR learning approach for AEC higher education - Implementation plan 


Institution Course (Academic Year 2023/2024) Expected attendees 
Western Sydney eo 
: . i e Advanced Design Communication (ARCH7007) 50 
University, Australia > Acne 
e ARCH7015 Practice Research Studio Civic (ARCH7015) 50 
University of Florence, ° BIM and Information Modeling of the Construction Process (B028836) 50 
Italy e Design and Safety of Workplaces B030584 (B063) 50 


The implementation of the proposed pedagogical model will engage the students in providing indications and 
opinions regarding the environment in which they are exploring and studying. This approach will help with 
validating both the pedagogical framework, as well as, the designed environments, by gathering the user’s 
experiences and observations. This, in turn, will assist in enhancing the entire framework developed in this study. 
For this purpose, and according to the principles discussed above, a prototypical questionnaire implementation 
comprising 16 questions has been developed and is represented in Figure 3. 


In addition to the questionnaire, a prototypical implementation of the student spatial tracking restitution in form 
of heatmap is proposed in Figure 4. As it can be seen, the more relevant areas of the virtual environment for 
learning purpose could straightforwardly be inferred based on the time spent, providing useful information in the 
development of further immersive teaching material (e.g., virtual environments). 
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SECTION A - EXTENDED REALITY TECHNOLOGIES IN CONSTRUCTION 


Student Data 


ID: 
Age: 
Course: 
Gender: 


Questionnaire 


1) Did you enjoy the experience? 
2) Were you comfortable during the experience? 
3) Was it easy to use the viewer and controller? 
4) Did it bother you not being able to see your body in the virtual world? 
5) Are the signal arrows helpful in figuring out where to go? 
6) Does the virtual building spaces sufficiently replicate the real one? 
7) Did you need less time to understand the content displayed in VR compared to a face-to- 
Contents' face power-point presentation? 
Effectiveness ) 8) Were your learning skills affected by the VR experience? 
9) Do you think this experience can replace the role of the educator? 
10) Do you think this experience is useful in getting to know a subject before it is explained 
to you by an educator ? 
11) Do you think this experience is useful in understanding the building and its technological 
Learning aspects? 


Immersivity 


Experience's 12) Is this type of learning useful for an inexperienced student? 

Effectiveness | 13) Is this type of learning useful for a student without any knowledge on the subject? 
14) Do you prefer to interact with an avatar or a teacher? 

Future 15) Would you like to listen audio explanations of the virtual scenarios? 

Development 16) Would you like to be able to interact with virtual objects? 


Suggestions 


Fig. 3: Immersive VR learning session evaluation questionnaire — Implementation prototype. 
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Fig. 4: Student spatial track heatmap representation — Implementation prototype 
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5. CONCLUSIONS 


This study, which was a successful collaboration between University of Florence and Western Sydney University 
aimed to provide a comprehensive understanding of student’s perception and demand of the newly developed 
pedagogical framework. This framework aimed to create a more engaging learning environment, while responding 
to the industries needs and requirements, and prepare students for the future of their careers in AEC. To achieve 
this, this study was developed in two stages. In stage one, a qualitative approach was used to develop a novel 
pedagogical framework, based on the constructivism method, followed by a problem-based approach and 
immersive learning method. To test the effectiveness of this developed model and student’s perception of it, a 
mixed-method approach was used to develop an in-depth questionnaire. Furthermore, a contextualized heatmap 
recording is accompanying the developed survey to ensure the robustness of the results of this study. 


The outcome of this research will be tested internationally at the architecture and civil engineering master classes 
which will be involved both in Australia (Western Sydney University) and Italy (University of Florence) in early 
2024. The outcome of the experiment will form another joint publication and collaboration between the two 
institutions. 
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